System and method of capturing and producing biometric-matching quality fingerprints and other types of dactylographic images with a mobile device

ABSTRACT

An optical module includes a housing with a lighting mechanism, an aperture formed therein, and a window that frames a transparent surface adapted to contact a skin surface of a person, for example, a fingertip. The lighting mechanism provides light to illuminate the skin surface placed upon the transparent surface. A prism has a first side facing the lighting mechanism, a second side at the window, and a third side through which a dactylographic image exits the prism. One or more light reflecting surfaces are disposed within the housing to reflect the dactylographic image towards the housing aperture. The optical module is coupled to a mobile device having a camera, with the aperture of the housing aligning with a lens of the camera. The camera acquires the dactylographic image, and the mobile device adjusts this dactylographic image to produce a dactylographic image suitable for biometric matching.

RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 61/590,836, filed Jan. 26, 2012, titled “System andMethod of Capturing and Producing Biometric-Matching QualityFingerprints through a Mobile Device”, and to U.S. ProvisionalApplication No. 61/682,346, filed Aug. 13, 2012, titled “System andMethod of Capturing and Producing Biometric-Matching QualityFingerprints through a Mobile Device”, the entireties of whichapplications are incorporated by reference herein.

FIELD OF THE INVENTION

The invention relates generally to dactylographic systems. Morespecifically, the invention relates to systems and methods of capturingand producing biometric-matching quality dactylographic images with amobile device.

BACKGROUND

Fingerprints are well known for their use in personal biometricidentification and verification applications. Fingerprinting can oftenbe encumbered by having to bring the person to be fingerprinted to afingerprinting station. This process can be inconvenient and timeconsuming. Fingerprinting “in the field” is thought to be a potentialalternative by which to perform the task in real time, thereby avoidingthe inconvenience and the consequent time delay.

SUMMARY

In one aspect, the invention features an optical module comprising ahousing with a lighting mechanism, a side with an aperture formedtherein, and a window framing a transparent surface used to contact askin surface of a person. The lighting mechanism is adapted to providelight that illuminates the skin surface of the person placed upon thetransparent surface. The optical module further comprises a prism havinga first side facing the lighting mechanism, a second side facing thewindow that frames the transparent surface used to contact the skinsurface of the person, and a third side through which a dactylographicimage of the skin surface of the person exits the prism. A plurality oflight reflecting surfaces is within the housing, spatially separatedfrom the prism. The plurality of light reflecting surfaces is disposedwithin the housing to reflect the dactylographic image of the skinsurface of the person exiting the prism towards the aperture in the sideof the housing.

In another aspect, the invention features a method for acquiring adactylographic image comprising: acquiring, by a camera of a mobiledevice, a dactylographic image of a skin surface of a person, andadjusting, by the mobile device, the acquired dactylographic image toproduce a dactylographic image of a quality suitable for biometricmatching.

In still another aspect, the invention features a mobile devicecomprising a camera, memory storing program code that, when executed,processes images of skin surfaces of people acquired by the camera, anda processor configured to execute program code stored in the memory inresponse to the camera acquiring a dactylographic image of a skinsurface of a person. The processor, in response to executing the programcode, adjusts the acquired dactylographic image of the skin surface of aperson to produce a dactylographic image of a quality suitable forbiometric matching.

In yet another aspect, the invention features a dactylographicimage-capture system comprising a mobile device with a native camerahaving a lens, memory storing program code that, when executed,processes dactylographic images of skin surfaces of people acquired bythe camera, and a processor configured to execute program code stored inmemory in response to the camera acquiring a dactylographic image of askin surface of a person, the processor, in response to executing theprogram code, adjusting the acquired dactylographic image of the skinsurface of the person to produce a dactylographic image of a qualitysuitable for biometric matching.

The dactylographic image-capture system further comprises an opticalmodule that is coupled to the mobile device. The optical modulecomprises a housing with a lighting mechanism, a side with an apertureformed therein, and a window framing a transparent surface used tocontact the skin surface of the person. The lighting mechanism isadapted to provide light to illuminate the skin surface of the personplaced upon the transparent surface. The optical module furthercomprises a prism having a first side facing the lighting mechanism, asecond side facing the window that frames the transparent surface usedto contact the skin surface of the person, and a third side throughwhich a dactylographic image of the skin surface of the person exits theprism. The optical module further comprises a plurality of lightreflecting surfaces within the housing spatially separated from theprism. The one or more light reflecting surfaces are disposed within thehousing to reflect the dactylographic image of the skin surface exitingthe prism towards the aperture in the side of the housing. The lens ofthe native camera of the mobile device aligns with the aperture in theside of the housing to receive the dactylographic image of the skinsurface passing therethrough.

In yet another aspect, the invention features an optical modulecomprising a housing with a diffuser window adapted to diffuse ambientlight, a frame portion with an aperture, and a second window framing atransparent surface used to contact a skin surface of a person. Theframe portion is adapted to couple to a mobile device with a built-incamera such that the aperture of the frame portion aligns with a lens ofthe camera of the mobile device. A prism is disposed in the housing witha first side of the prism at the second window and a second side of theprism facing the diffuser window. The diffused ambient light enteringthe housing through the diffuser window illuminates the skin surface ofthe person that is in contact with the transparent surface. A lens isdisposed within the housing to focus a dactylographic image of the skinsurface that exits the prism and propagates towards the aperture in theframe portion.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further advantages of this invention may be betterunderstood by referring to the following description in conjunction withthe accompanying drawings, in which like numerals indicate likestructural elements and features in various figures. The drawings arenot necessarily to scale, emphasis instead being placed uponillustrating the principles of the invention.

FIG. 1 is a block diagram of an embodiment of a dactylographic imageacquisition system including an optical module coupled to a mobiledevice.

FIG. 2 is a flow chart of an embodiment of a process for automatic andreal-time capture, processing, and use of a high-quality dactylographicimage in an example biometric matching application.

FIG. 3 is a side view of an embodiment of the optical module.

FIG. 4 is an isometric view of the embodiment of the optical module ofFIG. 3 coupled to a mobile device.

FIG. 5 is a side view of another embodiment of the optical modulewithout the housing and various internal components.

FIG. 6 is a side view of the embodiment of the optical module of FIG. 5with its housing.

FIG. 7 is a diagram illustrating paths taken generally by light passingthrough the optical modules of FIG. 3 and FIG. 5.

FIG. 8 is an exploded view of the optical module of FIG. 5.

FIG. 9 is a top down view of the optical module of FIG. 5.

FIG. 10 is a side view of the optical module of FIG. 5.

FIG. 11 is an isometric front view of the optical module of FIG. 5.

FIG. 12 is a top view of the optical module of FIG. 5 coupled to amobile device.

FIG. 13 is a bottom view of the optical module of the FIG. 5 coupled tothe mobile device.

FIG. 14 is a top view of the optical module of FIG. 5 coupled to amobile device from the perspective of the individual administering thefingerprinting.

FIG. 15 is a bottom view of the optical module of the FIG. 5 coupled tothe mobile device from the perspective of the individual beingfingerprinted.

FIG. 16A is a diagram illustrating a top view of another embodiment ofan optical module with a transparent surface.

FIG. 16B is a side view of the optical module of FIG. 16A, wherein theend with the transparent surface has a sloped side.

FIG. 16C is the optical module of FIG. 16A from the sloped end with thetransparent surface.

FIG. 16D is a bottom view of the optical module of FIG. 16A, with adiffuser window at the sloped end of the optical module.

FIG. 16E is a diagram showing a mobile device, having a native camera,coupled to one side of the optical module of 16A.

FIG. 17A is a diagram of a top view of another embodiment of an opticalmodule housing having a prism at one end, an interior lens, and a mobiledevice attachment mechanism at the opposite end.

FIG. 17B is a diagram of a side view of the optical module housing withan interior mirror for directing ambient light into the prism.

FIG. 17C is a diagram of an end view of the optical module housing witha diffuser window adjacent to a window that frames a transparent surfaceused for making contact with a skin surface of the user.

FIG. 18A is a diagram of a top view of another embodiment of an opticalmodule.

FIG. 18B is a diagram of a side view of the housing of the opticalmodule of FIG. 18A.

FIG. 18C is a diagram of the housing viewed from the sloped end having awindow that frames a transparent surface used for making contact with askin surface of the user.

FIG. 18D is a diagram of the housing viewed from the end opposite thesloped end, the housing having an interior mirror and lens appearing inphantom.

FIG. 18E is a diagram of a bottom view of the housing, with a diffuserwindow at the sloped end of the housing.

FIG. 18F is a diagram of a mobile device, having a native camera,coupled to the bottom side of the housing.

DETAILED DESCRIPTION

The optical modules, software system, and processes described hereinfacilitate convenient, “in the field” fingerprint-based enrollment,identification, and verification operations using a mobile device. Themobile device is configured with a software system that is used inconjunction with an optical module to capture and process fingerprintimages and to produce therefrom high-quality fingerprint images suitablefor biometric matching. Because the capture of fingerprint images canoccur in a minimally controlled lighting environment, the softwaresystem running on the mobile device applies certain image processingtechniques to improve the quality of the image and minimize false matchand non-false match rates. To compensate for the uncontrolled,non-homogeneous lighting of the fingerprint, one or more fingerprintimages may be acquired, adaptively enhanced, and merged to create asingle illumination-normalized fingerprint image. Various opticalmodules described herein used with a mobile device are lightweight (interms of ounces) and small, typically measuring less than four inches inlength, approximately one inch in height, and approximately one inch inwidth.

Although described herein predominantly with use in the acquisition andprocessing of fingerprint images, the described embodiments of opticalmodules and software system can be adapted to acquire various types ofdactylographic images which include, but are not limited to, one ormore, full or partial fingerprints, thumbprints, palm prints, orfootprints (e.g., plantar prints). For purposes of readability,references herein made to a dactylographic image encompass each of thesetypes of dactylographic images. Also for purposes of readability, thoseprinciples described in this detailed description in connection withfingerprints shall be understood to apply also to the other types ofdactylographic images.

FIG. 1 shows a functional block diagram of an embodiment of adactylographic image acquisition system 10 comprised of an opticalmodule 12 coupled to a mobile device 14 having one or more built-in(i.e., native) cameras 16. The optical module 12 can be designed fordetachable attachment to any of a variety of mobile devices having oneor more cameras. Examples of mobile devices for which the optical module12 can be customized, as described herein, include smart phones (e.g.,the Apple iPhone™, RIM Blackberry™, Samsung Galaxy™ phones, and GoogleAndroid™ phones), tablets (e.g., the Apple iPad™), and portable mediaplayers (e.g., Apple iPod Touch™). Means for attachment include a mobiledevice frame, fastened or mounted to the optical module 12, into whichthe mobile device 14 can snap or slide and be securely held in placeduring the photographing of the fingerprint as described herein.

The optical module 12 operates as a finger-to-camera interface andincludes a lighting mechanism 18 for illuminating the skin surface(e.g., fingertip) of the user during the fingerprint capture process. Inone embodiment, the optical module 12 is a passive device, meaning theoptical module 12 needs no electrical power for its operation, whereinthe lighting mechanism 18 comprises a diffuser window that allowsdiffused ambient light to enter the optical module 12. In anotherembodiment, the lighting mechanism 18 requires electrical power, forexample, one or more LEDs. In this embodiment, the optical module 12 hasa power source 20, for example, one or more solar cells or one or morebatteries.

In addition to the one or more cameras, the mobile device 14 compriseshardware 22, including memory 24, a processor 26, a network interface28, and a software system 30 that runs on the mobile device 14 toprocess captured fingerprint images and to produce therefromhigh-quality fingerprints suitable for biometric matching. Without theprocessing performed by the software system 30, the fingerprintscaptured by the camera of the mobile device would be unsuitable forbiometric matching. For one, the native camera system of the mobiledevice 14 is not designed to produce monochromatic close-up images offingerprints (for example) at 500 ppi to 1000 ppi resolution, whereasthe software system 30 is designed towards this end.

The memory 24 stores the program code associated with the softwaresystem 30 and the processor 26 executes the program code to accomplishthe fingerprint capture process, as described herein. The networkinterface 28 enables the mobile device 14 to communicate with a computersystem 32, for purposes of downloading data from the computer system 32or uploading data to the computer system 32 as described herein. Suchcommunication can be over a wired or wireless communication links.Example embodiments of the wireless communication link include, but arenot limited to, satellite communication links and radio frequency (RF)links.

In brief, the software system 30 includes an illumination normalizationalgorithm 34 to correct for the non-homogeneous lighting produced by thenatural environment in which a snapshot of the fingerprint is taken, andan image warping and scaling algorithm 36 to compensate for distortionsdue to perspective, magnification, and camera barrel distortion. Morespecifically, the image-warping and scaling algorithm 36 includes aperspective adjustment module 38, which compensates for the perspectivedistortion caused by the angled prism 88 (e.g., FIG. 3) of the unpoweredoptical module 12 used to acquire the fingerprint, a scaling andresolution adjustment module 40, which compensates for the magnificationperformed by the lens 92 (e.g., FIG. 3), and a barrel distortionadjustment module 42 to compensate for barrel distortion andmagnification introduced by the camera 16 in the mobile device 14. Othersoftware modules of the software system 30 can include a biometrictemplate extractor 44, a code reader 46, and a biometric matcher 48.

A calibration process determines parameters used by the image-warpingand scaling algorithm 36 during the image processing. Such parametersdepend upon the design of the optical module 12 (e.g., the distance of amirror from the lens) and on the type of mobile device (e.g., the cameralens location on the mobile device, focal length, magnification andbarrel distortion).

During the calibration process, the camera 16 of the mobile device 14captures an image from a predetermined calibration object (using theparticular optical module designed for that model of mobile device).This image captured by the camera of the mobile device is subsequentlycompared with a calibrated image (preloaded on the mobile device);certain points in the captured image are compared with correspondingpoints in the calibrated image. From this comparison, the softwaresystem 30 generates and stores a configuration file of parameter valuesin the memory 24 of the mobile device 14, to be used in the processingof subsequently captured fingerprint images.

The calibration process can also compensate for image reproductionvariance caused by mobile device manufacturing inconsistencies, bymaking use of data from a statistical analysis of the mobile devicemanufacturing error to determine an optimal image acquisition process.

The calibration process can be run once for each type of mobile device(e.g., at manufacture, one configuration file for each brand of Android™smart phone, another configuration file for iPhones™, etc.). Thenecessary calibration configuration file can then be automaticallydownloaded from a web service during setup (to ensure an up-to-dateversion), or supplied with the mobile device, to be loaded onto themobile device during setup. Alternatively, users can calibrate their ownmobile devices individually, prior to first use of the mobile device forcapturing a fingerprint image.

The result from the image acquisition is one or more processedmonochromatic fingerprint images at a 500 ppi or 1000 ppi resolution(depending on the biometric matcher requirements). Each resultingfingerprint image is thus suitable for biometric matching applications.

The dactylographic image capture system 10 for capturing and producinghigh-quality fingerprint images can be used in conjunction with otherbiometric applications. Such applications, in addition to collecting“live” fingerprints “in the field”, can also collect other optionaldata, including biographic information, acquire biometric samples,including face images and iris images. A camera of the mobile device(whether the same camera used to capture the fingerprints or, preferablya second camera facing the opposite side of the mobile device) canacquire these other biometric samples or read in biometric data from QRcodes or bar codes or some other graphical representation of biometricdata.

These other biometric applications include, but are not limited to:

1) Biometric verification, which is a process by which a single set(derived from a single person) of one (unimodal) or more (multimodal)live biometric samples is compared to a single “reference” set of one ormore biometric samples in order to ascertain their similarity and thuswhether they belong to the same person;2) Biometric identification, which is a process by which a single“probe” set of one or more live biometric samples is compared to a“gallery” of multiple sets of one or more biometric samples, in order toascertain which set in the gallery is most likely to belong to the sameindividual as that of the probe; and3) Biometric enrollment, which is a process by which biometric data suchas fingerprints, facial images, or iris images are collected using themobile device, along with biographic data such as name and contactinformation. This data is then either stored locally or transmitted by anetwork connection to a database containing a biometric gallery.

The biometric applications can involve any one or more of the followingmatching modes. These modes are differentiated from each other by wherethe reference biometric sample or biometric sample gallery resides andwhere the biometric comparison occurs.

1) Match-to-device. In this mode, the reference biometric sample orgallery of biometric samples resides on the mobile device 14. Thebiometric comparison is performed on the mobile device 14 and the useris informed of the comparison result through a software application userinterface running on the mobile device 14.2) Match-to-code. In this mode, the reference biometric sample isencoded in a QR (Quick Response) code or bar code that is either printedon a surface or displayed on a screen. A camera on the mobile device isused to take a photograph of the code and then the mobile device decodesthe biometric sample data from the photograph. A comparison is then madebetween the live biometric sample data and the data stored in the code.The comparison result is reported to the user through a softwareapplication user interface running on the mobile device 14.3) Match-to-chip. In this mode, the reference biometric sample isencoded on a chip, such as is found on a smart card used as acredential. The mobile device 14 uses near field communication (NFC) tocommunicate with the chip on the smart card and retrieve the referencebiometric sample. The mobile device 14 then performs a comparisonbetween the live sample data and the reference biometric sample. Thecomparison result is reported to the user through a software applicationuser interface running on the mobile device 14.4) Match-to-server. In this mode, the reference sample or gallery ofsamples resides on an external computer system (e.g., 32), preferablywithin physical proximity of the mobile device 14. The mobile device 14communicates with the computer system 32 over a network connectionestablished between the mobile device 14 and the computer system 32using, for example, a wireless technology such as GSM, wifi, orBluetooth. After establishing a network connection with the computersystem 32, the mobile device 14 retrieves either a reference sample orgallery and performs biometric comparison for verification oridentification at the mobile device 14. Alternatively, the mobile device14 submits the live probe sample to the computer system 32, where abiometric reference sample or gallery resides, and where the comparisonto the reference or gallery is performed. In this instance, the computersystem 32 can return the comparison result to the mobile device 14,which is then reported to the user through a software application userinterface running on the mobile device 14.

FIG. 2 shows an embodiment of a process 50 for automatic and real-timecapture and generation of a high-quality fingerprint image and use ofthe fingerprint image in an example biometric matching application. Inthe description of the process 50, reference is made to elements ofFIG. 1. At step 52, the mobile device 14 is calibrated (which, forexample, can occur during manufacturing or by the user). The camera 16of the mobile device 14 acquires (step 54) an image of fingerprint froma finger submitted to the optical module 12. The software system 30 ofthe mobile device 14 processes (step 56) the acquired fingerprint imageusing parameters determined during the calibration process, to produce ahigh-quality fingerprint image suitable for biometric matching.

In one embodiment, at step 54, the camera acquires multiple images ofthe fingerprint, each image being acquired with one or more differentfocal properties (e.g., focal length). Small changes in the focallength, for example, bring different features of the fingerprint intofocus. Each acquired image corresponds to a “slice” of the fingerprint.An image stacking technique then combines the slices in a manner thatselects the features in focus, and produces (step 56) a composite imageof the fingerprint.

The high-quality fingerprint image can be subsequently used inenrollment, verification, and/or identification processes. For example,in a verification application, the high-quality fingerprint image isencoded (at step 58) into a minutiae-based biometric template, and, fromthe biometric template, into a revocable biotoken. The software system30 applies a biometric template extraction algorithm 44 to extract abiometric template from each acquired live fingerprint image. Such datacorresponding to the high-quality fingerprint image is herein referredto as “live biometric sample data.” A camera 16 of the mobile device 14reads in (step 60) an image of a QR code or a bar code or graphicalrepresentation (the camera can read the code whether in print form ordisplayed on a device screen). Software 46 executing on the mobiledevice 14 decodes (step 62) the contents of the QR code, bar code, orgraphical representation and extracts (step 64) biometric data (e.g., abiometric template) from the decoded contents. Biometric matchingalgorithms 48 then perform (step 66) a comparison between the biometricdata extracted from the code and the live biometric sample data (ineither the biometric template or revocable biotoken form) in order toverify or authenticate the individual with respect to the QR code, barcode, or graphical representation.

FIG. 3 shows a side view of one embodiment of the optical module 12 thatcan be used in the acquisition of fingerprints as described herein. Theoptical module 12 includes a housing 80 with a lighting mechanism 18,also referred to as a diffuser window 18, and a window 82 that frames atransparent surface 70, which, in one embodiment, is one side of aprism. The diffuser window 18 diffuses the ambient light entering thehousing 80. In a preferred embodiment, the diffuser window 18 is made ofa rigid translucent material, such as glass or plastic. A rigidtranslucent diffuser window 18 has been found to sufficiently homogenizethe ambient light. (Non-homogenous lighting can reduce the quality ofcaptured fingerprint images, because non-uniform lighting can producefalse or mask true biometric features, rendering such resulting imagesproblematic for purposes of biometric matching). Another embodiment ofthe diffuser window 18 comprises a piece of translucent tape.

Instead of the diffuser window 18, other different embodiments can usesources of light other than ambient light, for example, one or more LEDspowered, for example, by one or more solar cells or a small battery. Inaddition, other embodiments can be adapted to capture fingerprints ofmultiple fingers simultaneously (e.g., a four-finger slap), using, forexample, multiple separate transparent finger surfaces or an enlargedsingle finger surface. Still other embodiments can have an singletransparent surface enlarged to contact skin surfaces for acquiringdactylographic images of palm prints or footprints. The side of thehousing 80 is shown to be transparent for purposes of revealing thevarious components within the housing; the transparency of the side isnot a requirement for the proper operation of the optical module 12.

The housing 80 encloses a first mirror (i.e., light reflecting surface)84, a prism 88, a second mirror (i.e., light reflecting surface) 90, alens 92, and a third mirror 94. The first mirror 84 is angled within thehousing 80 to direct ambient light entering the housing 80 through thediffuser window 18 towards one side 88 a of the prism 88. This ambientlight serves to illuminate the fingertip of the finger placed on thetransparent surface 70 (e.g., the exposed prism surface) framed by thewindow 82. Light carrying the image of the fingerprint leaves the prism88 through side 88 b. The second mirror 90 is angled within the housing80 to reflect the image of the fingerprint towards the lens 92. The lens92 is disposed between the second and third mirrors 90, 94, and focusesthe image of the fingerprint from the second mirror 90 onto the thirdmirror 94. The third mirror 94 is angled within the housing 80 toreflect the image of the fingerprint towards an aperture (e.g.,reference numeral 100 in FIG. 5). The attachment mechanism 96 holds themobile device 14 in place against the optical module 12 when capturingfingerprints.

FIG. 4 shows an example in which the mobile device 14 (with aback-facing camera, and optionally, a front-facing camera) is coupled tothe optical module 12 for capturing a fingerprint. The attachmentmechanism 96 has a slot 98 adapted to closely receive and hold one endof the mobile phone 14 such that the lens of the back-facing camera ofthe mobile device aligns with the housing aperture (e.g., referencenumeral 100 in FIG. 5) when the mobile phone 14 is coupled to theoptical module 12. Rather than directly couple to the mobile device, theattachment mechanism 96 may be designed to attach to a casing thatframes and holds the mobile device. Designing the attachment mechanismfor attachment to a casing can make the optical module 14 flexible foruse with different mobile device models.

Further, another embodiment can have the attachment mechanism 96 movablyattached to the mobile device casing so as to remove the optical module12 from the field of view of the camera aperture on the mobile device14, for example, on a rail extending left-to-right along the upper backregion of the mobile device. Moved in one direction along this rail, themobile device slides in place over the aperture; moved in the oppositedirection, the mobile device slides away from and out of the way of theoptical module 12. This movable attachment can facilitate joining theoptical module 12 to and removing the optical module 12 from the mobiledevice 14.

Another embodiment can have the attachment mechanism 96 movably attachedto the mobile device casing so as to remove the optical module 12 fromthe field of view of the camera aperture on the mobile device 14, forexample, on a pivotable arm that allows the optical module to twistaround an axis that is perpendicular to the back of the mobile device14. Twisted on this axis, the mobile device turns into place over theaperture; moved in the opposite direction, the mobile device is turnedout of the way of the optical module 12. This movable attachmentmechanism can facilitate joining the optical module 12 to and removingthe optical module 12 from the mobile device 14.

Yet another embodiment can have the attachment mechanism 96 movablyattached to the mobile device casing so as to remove the optical module12 from the field of view of the camera aperture on the mobile device14, for example, on an axle that allows the optical module twist aroundan axis that is parallel to the back of the mobile device 14. Twisted onthis axis, the mobile device is brought down into place over theaperture; moved in the opposite direction, the mobile device is flippedabove the mobile device 14 and out of the way of the optical module 12.This movable attachment mechanism can facilitate joining the opticalmodule 12 to and removing the optical module 12 from the mobile device14.

Provided the front-facing camera can attain sufficient resolution, otherembodiments of the optical module 12 can be adapted to use thefront-facing camera instead of the back-facing camera for capturingfingerprints. Alternatively, the front-facing camera can be usedsimultaneously with the back-facing camera; that is, the front-facingcamera can take a photograph of the user's face as an additionalbiometric, while the back-facing camera captures the user's fingerprint.

FIG. 5 shows a side view of another embodiment of the optical module 12that can be used in the acquisition of fingerprints. The housing 80, thefirst, second and third mirrors 84, 90, and 94, and the lens 92 havebeen omitted from the drawing to simplify the illustration of thepropagation of the fingerprint image through the optical module 12;their corresponding reference numerals being used here to indicate theirgeneral locations within the optical module 12. A planar section 102 ofthe housing frame has an aperture 100 formed therein. The image of thefingerprint of a fingertip pressed on the transparent surface 70 exitsthrough this aperture 100, as illustrated by light path 104.

FIG. 6 shows a side view of the embodiment of the optical module 12 ofFIG. 5 with its housing 80. The housing 80 is here shown as transparentfor purposes of displaying the internal components of the optical module12. Generally, the housing 80 is opaque, with the translucent diffuserwindow 18 attached thereto. The housing 80 also has a window 108 thatframes the transparent surface 70 of the prism 88, and a pair ofbrackets 110. Each bracket 110 has an opening 112. The brackets 110 arepart of an attachment mechanism for coupling the optical module 12 to acasing made for the mobile device 14.

FIG. 7 shows schematically the paths generally taken by light passingthrough the optical modules of FIG. 3 and FIG. 5. Ambient light,represented by arrow 120, enters the housing 80 through the diffuserwindow 18 and reflects off the first mirror 84 into the prism 88 andonto the interior side of the transparent surface 70, where itilluminates a fingertip placed on the exterior side of the transparentsurface 70. Light carrying the image of the fingerprint, represented bythe arrow 122, bends through the prism 88 towards the second mirror 90,which reflects the fingerprint image through the lens 92. The lens 92focuses the fingerprint image onto the third mirror 94, which reflectsthe fingerprint image into the aperture 100 (formed in the planarportion 102 of the housing frame).

FIG. 8 shows an exploded view of the embodiment of optical module 12 ofFIG. 5, including two housing halves 80-1, 80-2 that form the housing 80when joined together, and fasteners 130 for joining the halves 80-1,80-2 together. The diffuser window 18 has tabs 128 that couple to thehousing halves. The joined halves form an opening that aligns with theaperture 100 of the planar portion 102 of the housing frame. Thisopening closely receives a transparent exit window 132 made of glass orhardened clear plastic.

The mirror 84 is a diffuse mirror, and mirrors 90, 94 are fold mirrors.In one embodiment, the mirrors 84, 90 are rectangular in shape; themirror 94 is disc-shaped and fits in an opening (not visible) in asloped panel 134 of the housing frame, this opening being sized toclosely receive the mirror 94. This embodiment of attachment mechanism96 includes a spring latch 136, a torsion spring 138, and a latch axle140. When the attachment mechanism 96 is assembled, the latch axle 140passes through openings in the spring latch 136, the torsion spring 138,and the brackets 110 of the housing frame.

FIG. 9 shows a top view, FIG. 10, a side view, and FIG. 11, an isometricfront view of the optical module of FIG. 5. The isometric view in FIG.11 shows the spring-loaded latch 136, which is absent in FIG. 9 and FIG.10 (i.e., the brackets 110 are unattached). FIG. 12 shows a top view ofthe optical module 12 coupled to a mobile device 14. With the opticalmodule 12 coupled to a mobile device 14, the diffuser window 18 andmirror 84 do not impede view of the display of the mobile device 14.FIG. 13 shows a bottom view of the optical module 12 coupled to themobile device 14, wherein the optical module slides into rails of themobile device casing 150 and locks in place with the spring-loaded latch136.

FIG. 14 shows a top view of the optical module of FIG. 5 coupled to amobile device 14 from the perspective of an individual administering thefingerprinting, with the mobile device 14 being proximal and the opticalmodule 12 being distal with respect to the administrator. In thisorientation, the display of the mobile device 14 is visible to theadministrator, who can view the captured fingerprint image and anybiometric comparison results in real time.

FIG. 15 is a bottom view of the optical module of the FIG. 5 coupled tothe mobile device from the perspective of the individual beingfingerprinted, with the optical module 12 being proximal and the mobiledevice 14 being distal with respect to the individual beingfingerprinted. In this orientation, the transparent surface 70 faces theone being fingerprinted.

It is to be understood that the principles described herein extend tovarious other shapes of optical modules, with one, two, or more lightreflecting surfaces for defining the path taken by the light passingthrough the optical module housing. For example, FIG. 16A shows a topview of another embodiment of an optical module 12 used in theacquisition of fingerprints as described herein. Like the opticalmodules described in connection with FIG. 3 and FIG. 5, the opticalmodule 12 is designed for detachable attachment to any of a variety ofmobile devices having one or more native cameras. In this embodiment,the optical module 12 is a passive device, using ambient lightingthrough a diffuser window 178 (FIG. 16D) to illuminate a skin surfaceplaced on the transparent surface 170 during the dactylographic imagecapture process. This transparent surface 170 can be an exposed side ofthe prism 188 (FIG. 16B) within the housing 180, or a pane (e.g., glass)that abuts that side of the prism 188. Alternatively, illumination ofthe display of the mobile device 14 can be modulated to provide auniform illumination for the image acquisition. Shown in phantom in FIG.16A, to signify components within the housing 180, the optical module 12comprises a prism lens 160 and a plurality of light reflecting surfacesor mirrors 162-1, 162-2. The lens 160 may be an integral part of theconstruction of the prism 188.

FIG. 16B shows a side view of the optical module 12 of FIG. 16A. Fromthis side view perspective, the housing 180 is generally L-shaped with awindow 182 at the sloped end of the housing. A right-angle prism 188resides at the sloped end, with the hypotenuse side 164 a of the prism188 being exposed within the window 182 to serve as the transparentsurface 170 or, if the window 182 has a pane, abutting a transparentsurface of the pane. The lens 160 extends from another side 164 b of theprism 188. A third side 164 c of the prism 188 abuts the bottom thehousing 180.

The mirrors 162-1, 162-2 are arranged to reverse the direction of thelight as illustrated by arrow 168; the mirror 162-1 being directlyopposite the lens 160 and fixed at a 45-degree angle within the housing180 in order to reflect light coming through the lens 160, which carriesthe image of a fingerprint, towards the mirror 162-2. The mirror 162-2is at a 90-degree angle with respect to the mirror 162-1 in order tocomplete the 180-degree reflection. The particular placement of themirrors 162-1, 162-2 is configurable within each housing design anddetermined by the relative physical location of the camera lens of themobile device 14. The mirrors 162-1, 162-2 are disposed at preconfigureddistances from the lens 160, such distances from the lens 160 varyingfor different embodiments of the optical module 12, depending upon themodel (type) of mobile device 14 with which the optical module 12 isdesigned for use. The fingerprint image exits the optical module 12through an aperture 166 (FIG. 16C) in a side 172 of the housing 180.When positioned to capture fingerprint images, a mobile device 14 abutsthis side 172 of the housing 180, with the camera lens of the mobiledevice aligned with the aperture 166. An attachment mechanism holds themobile device 14 securely in place during the photographing of thefingerprint.

FIG. 16C shows the housing 180 from the sloped end with the transparentsurface 170. This end view shows the aperture 166 below the transparentsurface 170 (although, they are on different planes of the opticalmodule 12). The mirror 162-2 reflects light out through this aperture166.

FIG. 16D shows a bottom view of the housing 180, with a diffuser window178 at the sloped end of the housing 180, for purposes of diffusing theincoming ambient light. In a preferred embodiment, the diffuser window178 is made of a rigid translucent material, such as glass or plastic.In another embodiment, the diffuser window 178 is a piece of translucenttape placed over an opening in the housing at the third side 166 c (FIG.16B) of the prism 188. Other embodiments of the diffuser window 178 canbe fully transparent.

FIG. 16E shows an end view (opposite the sloped end) of a mobile device14 with a native camera coupled to one side of the housing 180. Aback-facing camera lens 190 of the mobile device 14 aligns with theaperture 166 opposite the mirror 162-2, which is within the housing 180.In this position, the front-side display of the mobile device 14 facesin the general direction of the diffuser window 178 and can thus be asource of light during image acquisition. Control of the illumination ofthe display during image acquisition may produce a more uniformillumination than ambient light alone.

To acquire a fingerprint, a subject presses his or her finger againstthe transparent surface framed by the window 182 at the sloped end ofthe housing. Ambient light enters the third side 164 c of the prism 188through the diffuser window 178 and illuminates the fingertip. The lens160 of the prism 188 focuses the image of the fingerprint onto themirror 162-1, the mirror 162-1 reflects the light onto the mirror 162-2,and the mirror 162-2 reflects the fingerprint image into the camera lens190. The camera system native to the mobile device 14 captures and savesthe fingerprint image. The software system 30 (FIG. 1) running on themobile device 14 can subsequently process the fingerprint image toproduce a high-quality image suitable for biometric matching.

FIG. 17A shows a top view of another example of a different design for ahousing 200 of an optical module 12. At one end of the housing 200 is aright-angle prism 202. At the end opposite the prism 202 is anattachment mechanism 204 for holding a mobile device 14. Disposed withinthe housing 200, between the prism 202 and the attachment mechanism 204,is a lens 206.

FIG. 17B shows a side view of the housing 200. A first side 208 a of theprism 202 abuts a window 210 with a pane that provides a surface for theplacement of one or more fingers. Alternately, the window 210 can beopen (i.e., a frame without a pane), and the first side 208 a of theprism 202 serves as a transparent surface for placement of thefinger(s). The housing 200 further includes an interior light reflectingsurface (e.g., a diffuse mirror) 212 for reflecting light into anotherside 208 b of the prism 202. The attachment mechanism 204 holds a mobiledevice 14 such that the lens 214 of a front-facing camera of the mobiledevice 14 faces the prism 202. (The mobile device 14 may be held in acasing designed specifically to fit closely in and couple to theattachment mechanism 204). The camera lens 214 aligns with an aperture(not shown) in the housing.

Ambient light enters the housing 200 through a translucent ortransparent diffuser window 216 (see also FIG. 17C), generally followingpath 218, reflecting off the light reflecting surface 212, passing intothe prism 202 through side 208 b, and illuminating the one or morefingers placed on the transparent surface framed by window 210. Theimage of the fingerprint(s) exits the prism 202 through its third side208 c and passes through the lens 206. The lens 206 focuses thefingerprint image at the aperture, behind which is disposed the lens 214of the front-facing camera of the mobile device 14. In accordance withthe principles described herein, the camera of the mobile device takes asnapshot of the fingerprint, and software running on the mobile deviceprocesses the image to produce a matching quality image.

FIG. 17C shows an end view of the housing 200 with the diffuser window216 adjacent to (or, in the figure, below) the transparent surface(i.e., window 210 or prism surface 208 a).

FIG. 18A shows a top view of still another example of a different designfor a housing 220 of an optical module 12. The housing 220 includes atransparent surface 222 (e.g., glass) at one end of the housing. Withinthe housing 220 is a mirror 224 at a preconfigured distance from a lens226. This distance varies for different embodiments of the opticalmodule 12, depending upon the model (type) of mobile device with whichthe optical module is designed for use. The particular placement of themirror 224 can be configured appropriately within the housing to suitthe type of mobile device and its relative physical location of thecamera lens.

FIG. 18B shows a side view of the housing 220, wherein the end with thetransparent surface 222 is sloped in shape. The housing 220 encloses aprism 228 disposed at the sloped end. One side 230 a of the prism 228can serve as the transparent surface 222. (Alternatively, thetransparent surface 222 can be a pane that abuts this side 230 a of theprism 228). The lens 226 is disposed at another side 230 b of the prism228 (the lens 226 may be an integral part of the construction of theprism 228). The mirror 224 is fixed at a 45-degree angle within thehousing 220 in order to reflect light coming through the lens 226 to oneside of the housing, as illustrated by arrow 232. A third side 230 c ofthe prism 228 abuts the bottom the housing 220. Where the third side 230c abuts, the housing 220 can have an open window so that the third side230 c is directly exposed to ambient light. This open window is coveredby a diffuser 234 (FIG. 18E).

FIG. 18C shows the housing 220 from the sloped end having thetransparent surface 222; FIG. 18D shows the housing 220 from the endopposite the sloped end, with the mirror 224 and lens 226 appearing inphantom.

FIG. 18E shows a bottom view of the housing 220, with a diffuser window234 at the sloped end of the housing 220. The diffuser window 234 can bemade of a translucent tape placed over an open window in the bottom ofthe housing and covering the exposed side 230 c of the prism 228. Thehousing 220 also has an elongated aperture 236 behind which the mirror224 is visible.

FIG. 18F shows a mobile device 14 with a camera coupled to the bottomside of the housing 220. The lens 238 of the mobile device's cameraaligns opposite the mirror 224 within the housing 220 to receive theimage of the fingerprint. To acquire a fingerprint, a subject presseshis or her fingertip against the transparent surface 222 at the slopedend of the housing 220. Ambient light enters the third side 230 c of theprism 228 through the diffuser window 234 and illuminates the fingertip.The lens 226 of the prism 228 directs the image of the fingerprint ontothe mirror 224, and the mirror 224 reflects the fingerprint image intothe camera lens 238. The arrangement is analogous to a simple periscopedesign, utilizing a lens and mirror to project the image onto a planeother than that of the camera lens 238. The camera system, native to themobile device 14, captures and saves the fingerprint image. In anotherembodiment, a powered light source (e.g., a flash or display screen ofthe camera, an LED) is used to illuminate the fingertip pressed on thetransparent surface 222 at the sloped end of the housing.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method, and computer programproduct. Thus, aspects of the present invention may be embodied entirelyin hardware, entirely in software (including, but not limited to,firmware, program code, resident software, microcode), or in acombination of hardware and software. All such embodiments may generallybe referred to herein as a circuit, a module, or a system. In addition,aspects of the present invention may be in the form of a computerprogram product embodied in one or more computer readable media havingcomputer readable program code embodied thereon.

The computer readable medium may be a computer readable storage medium,examples of which include, but are not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination thereof. As usedherein, a computer readable storage medium may be any tangible mediumthat can contain or store a program for use by or in connection with aninstruction execution system, apparatus, device, computer, computingsystem, computer system, or any programmable machine or device thatinputs, processes, and outputs instructions, commands, or data. Anon-exhaustive list of specific examples of a computer readable storagemedium include an electrical connection having one or more wires, aportable computer diskette, a floppy disk, a hard disk, a random accessmemory (RAM), a read-only memory (ROM), a USB flash drive, annon-volatile RAM (NVRAM or NOVRAM), an erasable programmable read-onlymemory (EPROM or Flash memory), a flash memory card, an electricallyerasable programmable read-only memory (EEPROM), an optical fiber, aportable compact disc read-only memory (CD-ROM), a DVD-ROM, an opticalstorage device, a magnetic storage device, or any suitable combinationthereof.

Program code may be embodied as computer-readable instructions stored onor in a computer readable storage medium as, for example, source code,object code, interpretive code, executable code, or combinationsthereof. Any standard or proprietary, programming or interpretivelanguage can be used to produce the computer-executable instructions.Examples of such languages include C, C++, Pascal, JAVA, BASIC,Smalltalk, Visual Basic, and Visual C++.

Transmission of program code embodied on a computer readable medium canoccur using any appropriate medium including, but not limited to,wireless, wired, optical fiber cable, radio frequency (RF), or anysuitable combination thereof.

The program code may execute entirely on a user's computer, partly onthe user's computer, as a stand-alone software package, partly on theuser's computer and partly on a remote computer or entirely on a remotecomputer or server. Any such remote computer may be connected to theuser's computer through any type of network, including a local areanetwork (LAN) or a wide area network (WAN), or the connection may bemade to an external computer (for example, through the Internet using anInternet Service Provider).

In addition, the described methods can be implemented on an imageprocessing device, fingerprint processing device, or the like, or on aseparate programmed general purpose computer having image processingcapabilities. Additionally, the methods of this invention can beimplemented on a special purpose computer, a programmed microprocessoror microcontroller and peripheral integrated circuit element(s), an ASICor other integrated circuit, a digital signal processor, a hard-wiredelectronic or logic circuit such as discrete element circuit, aprogrammable logic device such as PLD, PLA, FPGA, PAL, or the like. Ingeneral, any device capable of implementing a state machine that is inturn capable of implementing the proposed methods herein can be used toimplement the image processing system according to this invention.

Furthermore, the disclosed methods may be readily implemented insoftware using object or object-oriented software developmentenvironments that provide portable source code that can be used on avariety of computer or workstation platforms. Alternatively, thedisclosed system may be implemented partially or fully in hardware usingstandard logic circuits or a VLSI design. Whether software or hardwareis used to implement the systems in accordance with this invention isdependent on the speed and/or efficiency requirements of the system, theparticular function, and the particular software or hardware systems ormicroprocessor or microcomputer systems being utilized. The methodsillustrated herein however can be readily implemented in hardware and/orsoftware using any known or later developed systems or structures,devices and/or software by those of ordinary skill in the applicable artfrom the functional description provided herein and with a general basicknowledge of the computer and image processing arts.

Moreover, the disclosed methods may be readily implemented in softwareexecuted on programmed general purpose computer, a special purposecomputer, a microprocessor, or the like. In these instances, the systemsand methods of this invention can be implemented as program embedded onpersonal computer such as JAVA® or CGI script, as a resource residing ona server or graphics workstation, as a routine embedded in a dedicatedfingerprint processing system, as a plug-in, or the like. The system canalso be implemented by physically incorporating the system and methodinto a software and/or hardware system, such as the hardware andsoftware systems of an image processor.

It is, therefore, apparent that there has been provided, in accordancewith the present invention, methods for capturing and processingfingerprint images to produce high-quality fingerprints suitable forbiometric matching applications. While this invention has been describedin conjunction with a number of embodiments, it is evident that manyalternatives, modifications and variations would be or are apparent tothose of ordinary skill in the applicable arts. For example, an opticalmodule can be adapted to capture fingerprints without the use ofmirrors; for example, an elongated tube or rectangular housing with theprism at one end of the housing and the mobile phone with the camera atthe opposite end of the housing. Accordingly, it is intended to embraceall such alternatives, modifications, equivalents, and variations thatare within the spirit and scope of this invention.

Relative terms used herein, such as top, bottom, front, back, side,left, right, above, below, upper, and lower, refer to how features ofthe apparatus appear in the figures, and serve to facilitate thedescription of the invention, and are not meant to be interpreted aslimitations.

1. An optical module comprising: a housing with a lighting mechanism, aside with an aperture formed therein, and a window framing a transparentsurface used to contact a skin surface of a person, the lightingmechanism being adapted to provide light to illuminate the skin surfaceof the person placed upon the transparent surface; a prism having afirst side facing the lighting mechanism, a second side facing thewindow that frames the transparent surface used to contact the skinsurface of the person, and a third side through which a dactylographicimage of the skin surface of the person exits the prism; and a pluralityof light reflecting surfaces within the housing spatially separated fromthe prism, the plurality of light reflecting surfaces being disposedwithin the housing to reflect the dactylographic image of the skinsurface of the person exiting the prism towards the aperture in the sideof the housing. 2-55. (canceled)